Using Provenance to Streamline Data Exploration through Visualization

Scientists are faced with increasingly larger volumes of data to analyze. To analyze and validate various hypotheses, they need to create insightful visual representations of both observed data and simulated processes. Often, insight comes from comparing multiple visualizations. But data exploration through visualization requires scientists to assemble complex workflows—pipelines consisting of sequences of operations that transform the data into appropriate visual representations—and today, this process contains many error-prone and time-consuming tasks. We show how a new action-based model for capturing and maintaining detailed provenance of the visualization process can be used to streamline the data exploration process and reduce the time to insight. This model enables the flexible re-use of workflows, a scalable mechanism for creating a large number of visualizations, and collaboration in a distributed setting. A novel feature of this model is that it uniformly captures provenance information for both visualization data products and workflows used to generate these products. By also tracking the evolution of workflows, it not only ensures reproducibility, but also allows scientists to easily navigate through the space of workflows and parameter settings used in a given exploration task. We describe the implementation of this data exploration infrastructure in the VisTrails system, and present two case studies which show how it greatly simplifies the scientific discovery process.